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ABSTRACT 


It has been long accepted that students are themselves great resources when it comes to 
developing questions and activity guidelines. The present study utilizes a strategic 
understanding of how students can be encouraged to perform better in preparation for 
exams, by allowing them to frame their own subject wise questions. The application of drill 
and practice application (DPA) such as Peerwise and StudySieve focus on different batches 
of University students has revealed performance results that show us how to enhance 
examination performance. In this study, each student was required to use eitherfixed DPA 
(FDPA) or editable DPA (EDPA). The students used the assigned software for one whole 
semester. The treatment effectiveness was compared using students' final examination 
results. Results show that those students who were given the EDPA protocol scored better 
than those given the FDPA protocol of practice. There is a clear indication of better subject 
engagement in students who practiced using the EDPA application since they were allowed 
to edit and create the questions to which answers were to be written. In addition, students 
were shown to be able to understand subject contents better when they are responsible 
for generating items for their EDPA activities. 


Keywords: 


INTRODUCTION 

Drill and practice application (DPA) is a type of educational software that implements the drill and 
practice learning technique that is well associated with the behaviorist learning style. The major strengths of 
DPAs include allowing learners to master certain skills and concepts through repetitive activities 
accompanied with immediate feedback. Conventional use of DPAs only allow learners to drill on items made 
by content experts. This is useful for learners as the content developers highlight some of the important skills 
and concepts through carefully designed items. This type of DPA is normally embedded in various educational 
software packages for various types of audiences. Due to the rigid nature of items in this type of DPA, this 
study will refer to this DPA as Fixed DPA (FDPA). 

Currently, there is an evolving new type of DPA that allows learners to drill and review questions that 
they and their colleagues had submitted online. These more dynamic DPA attributes mark the migration of 
the behaviorist type to the more constructivist type DPA. Instead of just learning through answering the same 
questions, learners can enhance their learning by analyzing their present knowledge and subsequently create 
new and more meaningful DPA items. Learners can also discuss issues concerning the items with colleagues 
in an online collaborative working environment, hence improving item efficacy. This new type of DPA seems 
to be able to combine the strengths of behaviorism and constructivism into a more powerful learning 
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package. In addition, this type of learning environment seem to incorporate and promote other beneficial 
learning concepts including self-regulated learning, problem-based learning, community of practice and 
lifelong learning (Denny, Hamer, Luxton-Reilly & Purchase, 2008). 

Several working examples of this new type of DPA have shown positive results on student learning. An 
example is PeerWise which is an online DPA that utilizes student generated multiple-choice questions (MCQ); 
it allows students to contribute MCQ items for access by peers (Denny, 2013; Denny et al., 2008). Denny et 
al. (2008) reported that students' examination performance increased not merely due to answering and 
posting MCQ items but mostly due to the prolonged active use of PeerWise as a whole. Recently, PeerWise 
implemented a badge rewarding system to increase student participation (Denny, 2013). 

Another similar system known as StudySieve allows students to author free response type questions 
instead of just MCQs (Luxton-Reilly, Denny, Plimmer, & Bertinshaw, 2011). Luxton-Reilly et al. (2011) claimed 
that it is difficult to create MCQs that measure higher-order thinking skills (HOTS) which is normally easier 
with essay type questions (Paul & Nosich, 2009). The Luxton-Reilly et al. (2011) survey report also showed 
that students feel that they learned better when using StudySieve especially when they engaged in reviewing 
each other's items. 

Another system worth mentioning here is the Questions Sharing and Interactive Assignments 
(QSIA)(Barak & Rafaeli, 2004). Besides allowing students to contribute and review questions online like 
PeerWise and StudySieve, the QSIA also allows students to generate assignments which peers can complete 
either online or offline (Barak & Rafaeli, 2004). Barak and Rafaeli (2004) showed that students who 
contributed significantly in QSIA activities performed well in the examination compared to students who 
contributed less (p. 98). 

At the moment, these operational DPAs are targeted at students in higher learning institutions. Since 
this type of DPA has item editing features, this study will refer to this type of DPA as the Editable DPA (EDPA). 
Studies on these EDPAs have shown that students perform well in examinations when they actively 
participate in the EDPA activities as a whole compared to students who participated less (Barak & Rafaeli, 
2004; Denny et al., 2008). 

Studies on DPAs have yet to compare the effectiveness of a DPA when the items were made by content 
experts and by the learners. In fact, it is difficult to find such related studies. Therefore, this study would like 
to take the initiative to determine which drill type (EDPA or FDPA) is best for student learning. 

Purpose of Study 

The purpose of this study was to determine which is the best learning tool between EDPA and FDPA in 
helping students obtain better final examination results. The research question was: Is there a significant 
difference on final examination scores between EDPA and FDPA treatment groups? 

LITERATURE REVIEW 

Improving Knowledge Retention and Skills Through Drills 

Starting from physical exercises to professionally important skills, drills are known to be effective in 
promoting knowledge retention and skills. Studies have shown that when students were not subjected to 
continuous lectures and were stopped in between classes and asked to note their questions, they came up 
with new perspectives to the subject and also performed better in exams (Chin, 2001). Different teachers 
have been using this technique in different ways (Luik, 2007), and achieving the same result. This is actually 
an extension of the basic teaching principle which asks teachers to encourage doubts and questions from 
students after every new concept is introduced into the class. 

Learning Through Questioning 

The act of questioning lays the foundation for learning. Great scientists and thinkers have used 
questioning to develop their thoughts and discoveries (Boud, 1995). When this is encouraged amongst 
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university students, their learning becomes an active process. There is no doubt that students learn better 
through questioning. This makes students change from passive to active learners, and enable them to reason 
out questions which they would have found hard to answer otherwise (Chin, 2001). 

Self-assessment 

Self-assessment is important in promoting self-regulation to improve learning. This has been shown in 
the works of Alfke (1974) and Jelly (1985), who showed that while getting them to question was easy, 
watching students start self analyzing the answers they received was a rare occurrence. To bring them to the 
self-assessment mode, one needs to actively engage them in assessing answers received to their questions. 
In separate studies by Barak and Rafaeli (2004), we see that peer assessment becomes an interesting process 
for such students, which leads to self-assessment and improved learning (p. 88). 

Integrating with Computing Technology 

Computers can provide instantaneous feedback that can lessen the trouble for students flipping 
between the question and answer pages of a book. Computer drill and practice applications thus should be 
more useful than the conventional pen and paper drill and practice activities (Yu, 2009). Also, online 
discussions regarding the quality or difficulty level of the question, when fed in by peers who answer the 
questions, is effective in bringing about a growth of understanding in the subjects of the question papers. 

Many drill and practice applications are available in the market both for free and commercially 
available. Many DPAs seem not to be effective but some have shown good prospects with systems like 
PeerWise. Students have enjoyed the use of PeerWise and have developed better understanding of their 
subjects through it (Denny et al., 2008). 

DESIGN OF THE STUDY 

With the growing need to upgrade university education and assessment protocols to suit the speed 
and habits of Gen Y, we tried an experiment comparing examination performance of university students after 
using the DPAs. The EDPA version allowed the students to edit the questions while the FDPA version had 
fixed questions pre-set by the lecturer. The data were collected from the students' final examination 
performances. The stability of a thorough quantitative approach is relied upon in reaching the deductions 
shared at the end of this study. 

Sample of study 

The sample consisted of 151 students from two classes of the Cognitive Sciences and Ethics course 
taught by the same lecturer in Universiti Utara Malaysia. The students were from different semesters ranging 
from semester two to semester eight and enrolled in various programs such as Business Studies and 
Computing. They were required to participate for the entire duration of the semester from September 2013 
to December 2013. 

Treatment materials 

Students were given either one of the two versions of KAJI - an offline Windows based computer drill 
and practice application (DPA) using free-type response items built by the researcher. The first version is the 
Editable DPA (EDPA) while the second version is the Fixed DPA (FDPA). 

The main differences between EDPA and FDPA are the ability to edit items and the availability of pre¬ 
inserted items for drill and practice. EDPA has the ability to add and edit items through its own Question 
Manager (Figure 1). EDPA had no pre-inserted items. The items in Figure 1 are in the Malay language. 
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□ Question Manager |'-o- | 1 ® |m£3m| 

Course: SGDN1043 Sains Pemikiran dan Etika 


Topic: Pengenalan Falsafah, Pemikiran dan Etika 


No 

Question 

Answer 

Duration 

i 

Falsafah berkisartentang tiga ilmu hakikat utama ia.. 

1.Hakikat alam 

4 

2 

Berikan pengertian falsafah menurutl) Kamus Dew... 

Kamus Dewan mengatakan falsafah merupakan pengetahua... 

6 

3 

Berikan pengertian pemikiran 

Pemikiran bermaksud menggunakan akal untuk membuat a... 

1 

4 

Berikan pengertian etika 

Ukuran atau piawaian yang digunakan untuk membezakan ... 

1 

5 

Apakah perbezaan antara moral dan etika? 

Etika lebih bersifat teori yang masih boleh dipertikaikan, ma... 

2 

6 

Apakah hubungan antara falsafah, pemikiran dan ... 

Manusia menggunakan akal untuk berfikir tentang persoala... 

2 

7 

Teranqkan denqan rinqkas berkenaan falsafah dan ... 

a)Berpendapat etika merupakan suatu falsafah untuk men... 

8 






Add | j Edit | | Delete J j Preview | 


Figure 1. KAJI's Question Manager available only for the EDPA version. 

FDPA had no Question Manager to add or edit items but it has pre-inserted items for drill. This means 
that students treated with EDPA will need to generate items for their drills whereas FDPA students only need 
to drill on items already available in the application. A screenshot of a drill session is shown in Figure 2. 



Figure 2. A drill session in KAJI. 

The author used self developed applications was firstly to minimize extraneous variables that could 
affect results of study due to the use of different applications and secondly, self-developed applications 
would be more feasible to be maintained. The author opted for offline because many students lack a stable 
Internet connection in their rooms. In addition, KAJI can be installed using a copy and paste technique; thus 
it does not need any installers which will make it fixed to a single computer. KAJI can also be easily run from 
a pen drive or a different computer altogether. 

Assessment Material 

The study used the final examination results as the measurement of EDPA and FDPA effectiveness. 
Based on information from the lecturer, items were in the form of multiple choice questions (MCQs) and 
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were tested for validity and reliability. However, the information regarding the items was confidential as the 
items were difficult to develop and would be used in the near future. 

Experimental Procedure and Data collection 

During the first class after the Add and Drop week, students were given a homogeneity test to ensure 
their level of knowledge on the subject matter was similar. The test items used were past examination items 
and thus were tested in terms of validity and reliability. The information regarding the items were concealed 
as the items had the same importance as the incoming final examination items. An independent groups t- 
test was used to get the Levene's test probability value to test for homogeneity. The Levene's test indicated 
that the students' scores were homogenous (F = 1.79, p > .05). Students were then given KAJI and were 
briefed by their lecturer on how to use it during the first week of the semester. Class A was given EDPA while 
Class B was given FDPA. EDPA students must ensure that most of the important facts for each topic are 
covered through their items. 

Students were required to submit a report every week to prove that they have used KAJI at least twice 
a week. The report contains information about the items' details including how many times and when was 
the last time an item was attempted. The data collection process ended once the students' final reports and 
the final examination scores were collected from the lecturer. 

Data analysis 

Two groups of students from the same course were treated with two different version of DPAs. The 
treatment effect was tested using the students' examination scores. In other words, this experiment involved 
a set of scores that were collected from two different groups of participants coming from the same 
population. Therefore, the independent-groups f-test was chosen as the preferred method for data analysis 
to compare the differences of final examination scores between FDPA and EDPA conditions (see Coakes, 
Steed, & Ong, 2010). SPSS version 15 was used to perform the t -test calculations. The study used a 95% 
confidence intervals (Cl) for the mean difference. 


RESULTS AND DISCUSSIONS 

The results as in Table 1 show that there was a statistically significant difference in the examination 
scores for FDPA (M = 54.89, SD = 5.68) and EDPA (M = 56.92, SD = 5.87) conditions; t (149) = -2.15, p < .05. 
This means that based on the statistics, students who studied using EDPA had better scores in their final 
examination compared to students who used FDPA. 

Table 1 T-test Results for Comparing the Effect of EDPA and FDPA on Students' Examination Scores 


Type of DPA 

n 

Mean 

SD 

T 

df 

Sig. 

FDPA 

76 

54.89 

5.68 

-2.15 

149 

.033* 

EDPA 

75 

56.92 

5.87 





*p < .05 


There is still a possibility that students in the EDPA group may have been initially better than the FDPA 
group before the treatments were given. This argument is based on the fact that although both groups were 
shown to be initially homogenous at the knowledge level, the individual learning capabilities of each student 
could not be determined. Thus, this study can only assume that the learning capacity of all students involved 
were also homogenous. 
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CONCLUSIONS AND RECOMMENDATIONS 

Regardless of how hard the question paper was in the final examination, students who had used EDPA 
scored better than students who had used FDPA. This brings to the fore a very significant factor in active 
learning protocols used in universities till date. Students have the ability to innovate and improvise upon 
what is provided to them and when they do so, they are more inclined to understanding the subject better 
and helping each other understand it in a better way. With mutual help and better interaction, not just 
individual students (Zohar & Dori, 2003), but the entire group who attend to the sessions, benefit from the 
discussions and the whole class performs well (Williams, 1992). The performance indicates that FDPA 
protocols are similar to the print Question Answer exercises, although the level of engagement is higher. 
Therefore, EDPA is a more suitable form of DPA for the future which can be suggested to universities and 
colleges, and emulated with application software like StudySieve (Luxton-Reilly et al., 2011). After this study, 
it is also safe to say that KAJI can also be utilized in universities at mass level applications. This study would 
like to attempt a similar experiment in the future making changes only to the EDPA by providing it with pre¬ 
inserted items similar to those provided in the FDPA. 
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